W4. Fundamental Subspaces

Author

Salman Ahmadi-Asl

Published

February 13, 2026

1. Theory

1.1 Four Fundamental Subspaces of a Matrix

Every matrix defines four important vector subspaces that reveal its complete algebraic structure. Understanding these subspaces is essential for solving systems of equations, understanding matrix transformations, and applications in data science and engineering.

For an matrix with , there are four fundamental subspaces:

1. Column Space (or )

The column space is the set of all possible linear combinations of the columns of :

It is a subspace of (the “output space”)
Its dimension is (the rank)
The system has a solution if and only if
Basis: The pivot columns of the original matrix (identified from the row echelon form)

2. Null Space (or )

The null space is the set of all vectors that maps to zero:

It is a subspace of (the “input space”)
Its dimension is (the number of free variables)
It contains all solutions to the homogeneous system
Basis: The special solutions found by setting each free variable to 1 (and others to 0)

3. Row Space (or )

The row space is the span of the rows of , equivalently the column space of :

It is a subspace of
Its dimension is (the same as the column space)
Basis: The nonzero rows of the row echelon form (REF) of
The row space contains all linear combinations of the rows of

4. Left Null Space (or )

The left null space is the null space of :

It is a subspace of
Its dimension is
It is called “left” null space because is equivalent to (y multiplies A from the left)
Basis: Found by solving using row reduction on

1.1.1 Summary Table

Subspace	Notation	Lives in	Dimension	How to Find Basis
Column Space				Pivot columns of
Null Space				Special solutions of
Row Space				Nonzero rows of
Left Null Space				Special solutions of

1.1.2 The Rank-Nullity Theorem

The dimensions of these subspaces are not independent. Two fundamental equations connect them:

This simplifies to:

These follow from the Rank-Nullity Theorem: for any matrix, the rank plus the dimension of the null space equals the number of columns.

1.1.3 Computing the Four Subspaces: Step-by-Step

Example Setup: Let (a matrix).

Step 1: Row reduce to REF

The pivots are in columns 1 and 3. Thus .

Step 2: Column Space

The pivot columns in the original matrix form a basis:

Dimension: .

Step 3: Row Space

The nonzero rows of form a basis for the row space:

Dimension: .

Step 4: Null Space

Solve using :

Free variables: and . Setting gives . Setting gives .

Dimension: ✓

Step 5: Left Null Space

Solve . Form and row reduce:

Free variable: . The system becomes:

Setting : , . Thus:

Dimension: ✓

1.2 Orthogonal Complements

Two subspaces and of are orthogonal if every vector in is orthogonal to every vector in . This is denoted .

1.2.1 Definition of Orthogonal Complement

For a subspace , the orthogonal complement (read “V perp”) is the set of all vectors orthogonal to every vector in :

Key fact: is always a subspace. To check if , it suffices to verify that is orthogonal to a basis of (not every vector in ).

1.2.2 The Fundamental Theorem of Linear Algebra

The four fundamental subspaces come in orthogonal pairs:

Equivalently:

Why is this true? Consider , so . Let be any row of . Then:

This says that is orthogonal to every row of , i.e., . Conversely, if is orthogonal to every row, then .

1.2.3 Orthogonal Decomposition

The orthogonal complement relationship gives us a powerful decomposition: every vector in can be uniquely written as the sum of a component in a subspace and a component in its orthogonal complement.

Theorem (Orthogonal Decomposition): If is a subspace of , then every can be uniquely written as: where and .

We write this as:

The symbol denotes direct sum, meaning the spaces only overlap at .

For matrices, this gives two fundamental decompositions:

1.3 Properties of Orthogonal Complements

1.3.1 Property 1: Intersection is Zero

For any subspace , we have:

Proof: Suppose . Then and . Since , it is orthogonal to all vectors in , including itself:

Therefore .

1.3.2 Property 2: Dimensions Add to n

For any subspace :

Proof sketch: Let be an orthonormal basis for . Extend this to an orthonormal basis for (using Gram-Schmidt). The vectors are orthogonal to all , so they form a basis for . Thus .

1.3.3 Property 3: Double Complement Returns to V

For any subspace :

Proof: We show two inclusions.

(): Let . For any , we have by definition of . Thus is orthogonal to all vectors in , so .

(): By Property 2: , and . Thus . Since and they have equal dimension, they must be equal.

1.3.4 Property 4: Uniqueness of Decomposition

If where and , then and .

Proof: Rearranging:

The left side is in (subspace closed under subtraction), and the right side is in . The only vector in both is (Property 1), so:

1.4 Examples of Orthogonal Complements

1.4.1 Geometric Examples in

If is the xy-plane (the set ), then is the z-axis (the set ).
If is a line through the origin with direction vector , then is the plane perpendicular to .
If , then .
If , then .

1.4.2 Example in

Let .

To find , we need all vectors such that:

This gives:

Setting and :

Thus:

Note: ✓

1.5 Proving the Fundamental Theorem

1.5.1 Proof:

Theorem: For any matrix , .

Proof: We show both inclusions.

():

Let , so . We must show , i.e., is orthogonal to all vectors in .

Let . Then for some . Compute:

Thus for all , so .

():

Let , so for all . We must show .

In particular, is orthogonal to each row of (since the rows of are columns of ). Let be the -th row of . Then:

But this is exactly saying that (each entry of is ). Thus .

Conclusion: .

1.5.2 Proof:

This follows immediately from the previous result and Property 3:

Similarly, by applying the same arguments to :

1.6 Why Rank of A and are Equal

An important consequence of the Fundamental Theorem is that the row rank equals the column rank.

Theorem: .

Proof: From the Rank-Nullity Theorem applied to :

From the orthogonal complement relationship:

But , so:

Comparing with the first equation:

This means the number of independent rows equals the number of independent columns!

1.7 Least Squares Problems

When a system has no exact solution (inconsistent), we often want the “best approximate solution.” This leads to the least squares problem.

1.7.1 The Least Squares Problem Setup

Problem: Given (typically , more equations than unknowns) and , find that minimizes:

Why “least squares”? We’re minimizing the sum of squared errors:

where is the -th row of .

1.7.2 Geometric Interpretation

Since always lies in the column space , we want to find the point in that is closest to . This closest point is the orthogonal projection of onto , denoted .

The error vector is orthogonal to , so .

1.7.3 Why Least Squares?

Overdetermined systems () often have no exact solution because we have more constraints (equations) than degrees of freedom (unknowns). Common applications:

Data fitting: Fitting a curve or line to experimental data points
Regression analysis: Finding the best linear model for data
Signal processing: Estimating a signal from noisy measurements
Computer graphics: Approximating shapes with simpler models
Machine learning: Training models where we have many training examples

Instead of giving up, we find the that makes as close to as possible.

1.7.4 The Normal Equations (Preview)

The least squares solution satisfies:

These are called the normal equations. This equation says that the error is orthogonal to the column space of .

Derivation: The error is . For this to be the minimum error, must be perpendicular to , i.e., :

If has linearly independent columns (full column rank), then is invertible, and:

The matrix is called the pseudoinverse (or Moore-Penrose inverse) of .

2. Definitions

Column Space ( or ): The set of all linear combinations of the columns of ; a subspace of with dimension .
Null Space ( or ): The set of all solutions to ; a subspace of with dimension .
Row Space ( or ): The span of the rows of , equivalently the column space of ; a subspace of with dimension .
Left Null Space ( or ): The null space of ; a subspace of with dimension .
Rank: The dimension of the column space (or equivalently, row space) of a matrix.
Orthogonal Complement (): For a subspace , the set of all vectors in orthogonal to every vector in .
Orthogonal Subspaces: Two subspaces and are orthogonal if every vector in is orthogonal to every vector in .
Direct Sum (): The space of all vectors of the form where and , when .
Least Squares Problem: Finding the vector that minimizes .
Normal Equations: The system whose solution gives the least squares solution.
Pseudoinverse: For a matrix with full column rank, is the left inverse that computes the least squares solution.
Pivot Columns: Columns of the original matrix that contain pivots in its row echelon form; they form a basis for .
Free Variables: Variables corresponding to non-pivot columns; they parametrize the null space.
Leading Principal Minor: The determinant of the top-left submatrix of a matrix (used in LDL^T decomposition).

3. Formulas

Four Fundamental Subspace Dimensions: For an matrix with rank :
Rank-Nullity Theorem: and
Orthogonal Complement Relationships:
- and
- and
Dimension of Orthogonal Complement: for any subspace
Direct Sum Decompositions:
Normal Equations:
Least Squares Solution (when has full column rank):
Projection onto Column Space:
Projection Matrix: projects onto
LDL^T Decomposition (for symmetric ): where is unit lower triangular and is diagonal

4. Practice

4.1. Four Fundamental Subspaces of a Matrix (Lab 4, Task 1)

Given the matrix

Compute an echelon form of and its rank .
Find a basis for and for .
Find a basis for and for .
State the dimensions of the four fundamental subspaces , , , and , and verify that they satisfy where is the number of rows of and is the number of columns.

Click to see the solution

Key Concept: Row reduction reveals the rank and provides bases for all four fundamental subspaces simultaneously.

(a) Echelon form and rank:

Row reduce :

The echelon form is , with pivots in columns 1 and 2. Thus .

(b) Bases for and :

Basis for : The pivot columns of the original matrix (columns 1 and 2):
Basis for : The nonzero rows of the echelon form:

(c) Bases for and :

Basis for : Solve using the echelon form. Free variables are and .

From row 2: .

From row 1: .

Setting : . Setting : .

Basis for : Solve . Row reduce :

Free variable: . From row 2: . From row 1: .

(d) Dimension verification:

Answer: . The bases are given above; all dimension formulas are satisfied.

4.2. Orthogonal Complement of a Plane (Lab 4, Task 2)

Find the orthogonal complement of the plane spanned by the vectors and by taking these to be the rows of and solving . Remember that the complement is a whole line.

Click to see the solution

Key Concept: The orthogonal complement of the row space of equals the null space of (by the Fundamental Theorem of Linear Algebra). The plane is , so we need .

Set up the matrix:
Row reduce the augmented matrix :
Read off the null space: Free variable .

Answer: The orthogonal complement of the plane is the line (or equivalently ).

4.3. Orthogonal Complement of Row Space and Decomposition (Lab 4, Task 3)

Find a basis for the orthogonal complement of the row space of Split into a row space component and a nullspace component .

Click to see the solution

Key Concept: The orthogonal complement of is . To split , we express it as a combination of a vector in and a vector in .

Find a basis for :

Free variable : , . Basis: .

A basis for is and (from the RREF rows).
Decompose : Write :

From rows 1 and 2: , . Substituting into row 3: . Thus , .

Row space component:
Null space component:
Verify: ✓, and ✓.

Answer: Basis for : . Row space component: ; null space component: .

4.4. Why These Orthogonality Statements Are False (Lab 4, Task 4)

Why are these statements false?

If is orthogonal to , then is orthogonal to .
If is orthogonal to and is orthogonal to , then is orthogonal to .

Click to see the solution

Key Concept: Orthogonality is not transitive, and taking complements does not preserve orthogonality. Counterexamples in disprove both statements.

(a) Counterexample:

Let and in .

Clearly since .

Now and .

Both complements contain the vector , so and and .

Thus is not orthogonal to . The statement is false.

(b) Counterexample:

Let , , in .

: ✓
: ✓
But , so is not orthogonal to .

The statement is false because orthogonality is not transitive.

Answer: Both statements are false by the counterexamples above.

4.5. Vectors Orthogonal to the Fundamental Subspaces (Lab 4, Task 5)

Find a vector orthogonal to the row space of , a vector orthogonal to the column space, and a vector orthogonal to the nullspace.

Click to see the solution

Key Concept: A vector orthogonal to the row space is in the null space; a vector orthogonal to the column space is in the left null space; a vector orthogonal to the null space is in the row space (by the Fundamental Theorem).

Row reduce :

Pivot columns: 1 and 3. . Free variable: .

(orthogonal to row space): Setting : , .
(orthogonal to column space): Solve .

Free variable : , .

Row space (orthogonal to null space): Any nonzero row of the echelon form:

Answer: (orthogonal to row space), (orthogonal to column space), (orthogonal to null space).

4.6. Block Matrices: Echelon Form and Special Solutions (Lab 4, Task 6)

Find (echelon form) for each of these block matrices, and find the special solutions:

Click to see the solution

Key Concept: Row operations can be described for block matrices using the same Gaussian elimination. The block structure determines how pivots appear and what the special solutions look like.

Matrix ():

Echelon form . Pivots in columns 1 and 3; free variable: .

From row 2: .
From row 1: .
Setting : special solution .

Matrix ():

Row reducing is the same as row reducing , applied to both blocks:

Free variables: (columns 2, 4, 5, 6 are free). The RREF gives … More precisely: from the RREF of , pivots are in columns 1 and 3, so free columns are 2, 4, 5, 6. Setting each free variable to 1 in turn yields 4 special solutions.

Matrix ():

Subtract top block from bottom block ():

Echelon form of has the same structure, with rank . The null space has dimension .

Answer: with special solution for . For (rank 2), there are 4 special solutions. For (rank 4), there are 2 special solutions.

4.7. Construct a System from a Complete Solution (Lab 4, Task 7)

I. Find a system whose complete solution is

II. Find a system with these solutions exactly when .

Click to see the solution

Key Concept: The complete solution decomposes as a particular solution plus the null space. We reverse-engineer the matrix and right-hand side.

Part I:

Identify the particular solution and null space direction:
- Particular solution:
- Null space direction:
Build with : Use RREF with free variable : Verify: ✓
Compute :

Part II: Add a third row that is the sum of the first two (so consistency requires ):

Answer:

Part I:
Part II:

4.8. Complete Solutions of Linear Systems (Lab 4, Task 8)

Find the complete solutions of:

Click to see the solution

Key Concept: The complete solution is the sum of a particular solution and the general homogeneous solution. Row reduce the augmented matrix to identify pivot/free variables.

(a):

Row reduce the augmented matrix:
Back-substitute: From row 2: . From row 1: .
- Free variable : .
Complete solution:

(b):

Row reduce:

Free variables , ; pivot variables , .
Complete solution:

Answer:

(a):
(b):

4.9. Normal Vector and Orthogonal Complement of a Plane (Assignment 4, Task 1)

Let .

Find a nonzero vector normal to and verify that .
Find a basis for .
Find a basis for and relate it to .

Click to see the solution

Key Concept: A plane through the origin in defined by has normal vector , and every point in the plane is orthogonal to this normal.

(a) Normal vector:

The plane is . The coefficients give the normal vector:

Verification : A vector is in iff , which is exactly the defining equation of . ✓

(b) Basis for :

We need two linearly independent solutions of . Free variables: .

Set : . Vector: .
Set : . Or set : . Simpler: set : . Vector: .

Basis for : .

(c) Basis for :

Since has dimension 2 in , has dimension 1. We just found that is orthogonal to , so:

Answer: ; basis for : ; basis for .

4.10. Four Subspaces of a Parametric Matrix (Assignment 4, Task 2)

For , consider

Find bases for , , , and in terms of , , .
State explicitly the two orthogonality relations among the four subspaces.
Determine whether the dimensions of the four subspaces change if .

Click to see the solution

Key Concept: The matrix is already in row echelon form. We can read off pivots and free variables directly.

(a) Identifying subspaces:

The matrix is and already in REF. Pivots are in columns 2, 3, and 4. Thus .

: Pivot columns of the original matrix: columns 2, 3, 4.
: Nonzero rows of the REF (which is itself):
: Free variables are (columns 1, 5, 6). Set each to 1 in turn and solve back. This gives 3 special solutions.
: . Since the last row of is zero, satisfies :

(b) Orthogonality relations:

(c) Effect of :

If , the matrix becomes block diagonal with the same pivot structure. The rank remains , so all four subspace dimensions are unchanged: , , , .

Answer: regardless of . Dimensions are for Col, Row, Nul, Nul() respectively. Setting does not change any dimensions.

4.11. Four Subspaces of , , and (Assignment 4, Task 3)

Let Describe the four fundamental subspaces of each of the matrices , , and .

Click to see the solution

Key Concept: Compute the matrices explicitly, then determine their ranks and null spaces by row reduction.

Matrix :

Row reducing :

This is in REF. Pivot columns: 2 and 3. Rank: .

where (column 2 of ) and (column 3 of ). .
. .
: is free; special solution . .
: . Solve : . From row 3: ; row 2: ; free. .

Matrix :

This is upper triangular with 1s on the diagonal, so it is invertible ().

, , , .

Matrix :

Row reduce: pivots in columns 2 and 3. .

… Column 2 is , column 3 is . , .
, .
: free variable ; special solution . .
: . . free, . .

Answer: and both have rank 2, with null spaces and left null spaces . is invertible (rank 3), with trivial null spaces.

4.12. Is in or ? (First Matrix) (Assignment 4, Task 4)

Let Determine whether , whether , both, or neither.

Click to see the solution

Key Concept: Check by computing ; check by solving .

Check : Compute :

Since , .

Check : Solve by row reducing . If the system is consistent, .

Row reduce:

After Gaussian elimination (checking for consistency), the system is consistent (no pivot in the augmented column). Thus .

Answer: but .

4.13. Is in or ? (Second Matrix) (Assignment 4, Task 5)

Let Determine whether , whether , both, or neither.

Click to see the solution

Check : Compute :

Since , .

Check : Row reduce to check consistency.

After row reduction, if a row with appears, the system is inconsistent. Performing Gaussian elimination reveals an inconsistency, so .

Answer: and — neither.

4.14. Subspaces in vs (Assignment 4, Task 6)

Let be an matrix. Which of the following subspaces are in and which are in ? How many distinct subspaces can appear in this list (in general)?

Click to see the solution

Key Concept: The ambient space of each subspace is determined by where the vectors live — input space or output space .

In : , , — all subsets of the input space.
In : , , — all subsets of the output space.

Note: and , so the six entries reduce to four distinct subspaces: , , , .

In general, there are 4 distinct subspaces (the four fundamental subspaces).

Answer: , , live in ; , , live in . In general, 4 distinct subspaces.

4.15. Justifying the Rank-Nullity Identities (Assignment 4, Task 7)

Let be an matrix. Justify the identities:

Click to see the solution

Key Concept: These are the two statements of the Rank-Nullity Theorem applied to and .

First identity: .

The Rank-Nullity Theorem states that for any linear map ,

For as a linear map : , . But (row rank equals column rank). So:

Second identity: .

Apply the Rank-Nullity Theorem to :

Since :

Answer: Both identities follow directly from the Rank-Nullity Theorem applied to and , together with the fact that row rank equals column rank.

4.16. Equivalence: Full Column Space, Nul, Consistency (Assignment 4, Task 8)

Let be an matrix. Show that the following are equivalent:

is consistent for every .
.
.

Click to see the solution

Key Concept: These three conditions all say the same thing: every output vector is achievable, i.e., is surjective. We prove the equivalences in a cycle.

(a) (b): If is consistent for every , then every is in . Thus .

(b) (a): If , then every is a linear combination of the columns of , i.e., has a solution .

(b) (c): If , then , so . By the dimension formula: , so .

(c) (b): If , then , so , meaning . Since and has dimension , we get .

Answer: All three are equivalent; each follows from the others via the column space, rank, and Rank-Nullity Theorem.

4.17. Normal Vectors and Orthogonal Complement Construction (Assignment 4, Task 9)

Consider the planes in :

Find normal vectors to .
Find bases for and .
Construct a matrix whose row space is . Show that .
Construct a matrix whose column space is . Show that .

Click to see the solution

(a) Normal vectors:

(coefficients of )
or simplified (coefficients of )

(b) Bases:

Basis for : Free variables . Set : , so (multiply by 3). Set : , so (multiply by 3). Basis: .
Basis for : . Free: . Set : . Set : . Basis: .

(c) Matrix with row space :

Take a basis for as rows:

: solve . Row reduce: . One free variable gives a 1-dimensional null space. Setting : , . So . Thus . ✓

(d) Matrix with column space :

Take a basis for as columns:

$(A_2^T) = $ left null space of . . Find with :

From row 1: . From row 2: . Set : . ✓

Answer: , . The constructions confirm the null space relationships.

4.18. Four Subspaces from Row-Equivalent Matrices (Assignment 4, Task 10)

The matrices below are row equivalent:

Find and without further row reduction.
Find a basis for and a basis for .
Find a basis for .
Find a basis for .

Click to see the solution

Key Concept: Row equivalent matrices have the same row space and null space. The echelon form gives us everything we need.

(a) Rank and nullity:

has 2 nonzero rows, so . The matrix is , so .

(b) Bases for and :

: The nonzero rows of :
: Pivots of are in columns 1 and 2. Use the corresponding columns of the original :

(c) Basis for :

RREF from : and eliminate:

Free variables: . Special solutions:

: , . .
: , . .
: , . .

(d) Basis for :

. The last two rows of are zero, meaning rows 3 and 4 of are linear combinations of rows 1 and 2. To find , row reduce . The zero rows of correspond to the relations among rows of . From the row reduction of to , the multipliers give vectors in .

Answer: , . Row basis, column basis, and null basis are as computed above.

4.19. Four Subspaces and Orthogonality Verification (Assignment 4, Task 11)

Let Find bases for , , , and , and verify the two orthogonality relations.

Click to see the solution

Row reduce :

Swap :

All 3 columns are pivot columns. , so .

Bases:
- : The 3 pivot columns of form a basis:
- : Nonzero rows of REF: (as row vectors in ).
- (no free variables).
- : . Solve (i.e., row reduce ) to find 1 vector.
Orthogonality: is trivial since . : verify the basis vector of is orthogonal to each column of .

Answer: , , is 1-dimensional. The two orthogonality relations hold by the Fundamental Theorem.

4.20. Four Subspaces of a Matrix (Assignment 4, Task 12)

Let Find bases for , , , and , and verify the two orthogonality relations.

Click to see the solution

Row reduce:

Pivots in columns 1 and 2. . Free variables: .

Bases:
- : Columns 1 and 2 of original : .
- : .
- : Set : . . Set : . . Set : . .
- : . (since , has full row rank).
Orthogonality:
- : Verify e.g. ✓.
- : trivially satisfied since .

Answer: . Bases as given. Both orthogonality relations verified.

4.21. Dimensions of Four Subspaces Depending on Parameters (Assignment 4, Task 13)

Find the dimensions of the four fundamental subspaces of , depending on the parameters and :

Click to see the solution

Key Concept: The rank depends on the parameters. Row reduce and determine when rows become linearly dependent.

Row reduce:
Case analysis:
- Generic case ( and ): All 3 pivots exist. . Dimensions: , , , .
- Case , : Row 2 becomes . After swapping with row 3: pivots in columns 1 and 3. . Dimensions: , , , .
- Case , : Row 3 becomes zero. . Dimensions: , , , .
- Case , : Both conditions. Row 2 and row 3 become and . . (Or if row 2 becomes zero too — check: , not zero. So .)

Answer:

Condition
and	3	0	0
or (or both)	2	1	1

4.22. Row Reduce to Echelon Form and Identify Free Variables (Tutorial 4, Task 1)

Reduce and to echelon form, find their ranks, and identify the free variables:

Click to see the solution

Matrix ():

Row reduce:
Rank: . Pivots in columns 1 and 2. Free variables: and .

Matrix ():

Row reduce:
Rank: . Pivots in columns 1 and 2. Free variable: .

Answer:

: rank 2, free variables .
: rank 2, free variable .

4.23. Column Space of an Invertible Matrix (Tutorial 4, Task 2)

If is any invertible matrix, what is its column space? Why?

Click to see the solution

Key Concept: For an invertible matrix, the rank equals the number of rows (and columns), so the column space is the entire ambient space.

If is and invertible, then . Since and has dimension 8, we have:

Why? Because has a unique solution for every . This means every vector is in the column space.

Answer: , because has full rank 8 and its columns span all of .

4.24. Solvability Condition and Complete Solution (Tutorial 4, Task 3)

Under what condition on is the following system solvable? Include as a fourth column in . Find all solutions when that condition holds:

Click to see the solution

Key Concept: Augmented matrix row reduction reveals the solvability condition as a constraint on , and the free variables give the null space.

Row reduce :

Solvability condition: The last row requires , i.e.:
Complete solution (when the condition holds): Free variable .
- From row 2: .
- From row 1: .

Answer: Solvability condition: . When satisfied, the complete solution is .

4.25. Choose to Control Rank (Tutorial 4, Task 4)

Choose the number so that (if possible) the ranks are (a) 1, (b) 2, (c) 3:

Click to see the solution

Matrix ():

Row reduce:

Rows 1 and 2 are proportional, so the first two rows contribute only one pivot. Rank depends on :

(a) Rank 1: Need the third row to also be zero: .
(b) Rank 2: Need (so the third row gives a second pivot).
(c) Rank 3: Impossible — rows 1 and 2 are proportional, maximum rank is 2.

Matrix ():

Row reduce: :

(a) Rank 1: Need .
(b) Rank 2: Need .
(c) Rank 3: Impossible for a matrix (rank ).

Answer:

	Rank 1	Rank 2	Rank 3
			Impossible
			Impossible

4.26. Gram-Schmidt Process and QR Decomposition (Tutorial 4, Task 5)

Apply the Gram-Schmidt process to and write the result in the form .

Click to see the solution

Key Concept: Gram-Schmidt produces orthonormal columns , and is upper triangular.

: , . .
: Subtract projection of onto :
- .
- .
- . .
: Subtract projections of onto and :
- , .
- .
- . .
and :

Verify: ✓

Answer: , .

4.27. Symmetric Factorization (Tutorial 4, Task 6)

Compute the symmetric factorization of:

Click to see the solution

Key Concept: For a symmetric matrix, where is unit lower triangular (1s on diagonal) and is diagonal. The factorization is the symmetric version of .

Matrix :

Gaussian elimination, recording pivots and multipliers:

Pivot 1: . Multipliers: , .

After eliminating column 1:

Pivot 2: . Multiplier: .

After eliminating column 2:

Pivot 3: .
Result:

Matrix (symbolic):

Pivot 1: . Multiplier: . After elimination: .

Note: This requires and (i.e., is nonsingular).

Answer:

with , .
with , .

4.28. Subspace, Orthogonal Complement, and Decomposition (Tutorial 4, Task 7)

Find a basis for the subspace spanned by all solutions of .

Find a basis for the orthogonal complement .
Find and so that .

Click to see the solution

Key Concept: The subspace is the null space of the matrix . Its orthogonal complement is the row space of .

Basis for :

Solve . Free variables: .

(a) Basis for :

. .

(b) Decompose :

Write where and .

Since with :

Verify: ✓. Check : ✓. Check : ✓.

Answer:

Basis for : .
Basis for : .
, .

4.29. Construct a System with a Given Complete Solution (Test I Recap, Task 1)

I. Find a system whose complete solution is

Find a system with solutions exactly when .

Click to see the solution

Key Concept: The complete solution consists of a particular solution plus the null space. We need to reverse-engineer the matrix and right-hand side from this solution.

Part I: Find a system

Identify the particular solution and null space direction:
- Particular solution:
- Null space direction: (free variable )
Construct the matrix such that :

Verify: ✓
Find :

Part II: Find a system

Add a third row that is the sum of the first two:

Answer:

Part I:
Part II:

4.30. Find Complete Solutions of SLAEs (Test I Recap, Task 2)

Find the complete solutions of the following systems.

Click to see the solution

Key Concept: The complete solution is the sum of a particular solution and the general homogeneous solution.

Part (a):

Row reduce the augmented matrix:
Back-substitute: From row 2: . From row 1: . Free variable .
Complete solution:

Part (b):

Row reduce:
Free variables , ; pivot variables , .
Complete solution:

Answer:

(a):
(b):

4.31. Orthogonal Complement of Row Space (Test I Recap, Task 3)

Find a basis for the orthogonal complement of the row space of
Split into a row space component and a null space component .

Click to see the solution

Key Concept: The orthogonal complement of the row space is the null space (Fundamental Theorem). For part (b), decompose as a combination of row space and null space vectors.

Part (a):

Row reduce :

Free variable : , . Basis for : .

Part (b):

Basis for : , . Write :

From rows 1 and 2: . Substitute into row 3: .

Thus , and:

Verify: ✓. ✓.

Answer:

(a): Basis for :
(b): ,

4.32. Why These Orthogonal Statements Are False (Test I Recap, Task 4)

Why are the following statements false?

Statement 1: If is orthogonal to , then is orthogonal to .
Statement 2: If is orthogonal to , and is orthogonal to , then is orthogonal to .

Click to see the solution

Key Concept: These statements claim orthogonality relations that don’t hold. We disprove them with counterexamples in .

Statement 1:

Let , . Then .

, .

Both complements contain , and . So . False.

Statement 2:

Let , , .

: ✓
: ✓
; .
. So . False.

Answer: Both statements are false by the counterexamples above.

4.33. Gram-Schmidt Orthogonalization (Test I Recap, Task 5)

Using the following set of non-orthogonal vectors, determine a corresponding set of orthonormal vectors using the Gram-Schmidt process:

Click to see the solution

Key Concept: The Gram-Schmidt process creates orthogonal vectors by subtracting projections onto previously computed vectors, then normalizes to get an orthonormal set.

First vector:
Second vector: Subtract projection of onto :
- , .
- .
- . .
Third vector: Subtract projections of onto and :
- ; .
- ; .
- .
- . .

Answer:

4.34. Solve Using LU-Decomposition (Test I Recap, Task 6)

Solve the matrix equation using LU-decomposition and forward/backward substitution:

Click to see the solution

Key Concept: where is unit lower triangular and is upper triangular. Solve (forward), then (backward).

LU decomposition: Gaussian elimination on :
- , . After step 1: .
- . After step 2: .
- .
Forward substitution :
Backward substitution :

Answer: .

4.35. Check Symmetric Positive Definiteness (Test I Recap, Task 7)

Check whether the following matrices are symmetric positive definite:

Click to see the solution

Key Concept: A symmetric matrix is positive definite iff all its leading principal minors are positive (Sylvester’s criterion).

Matrix :

is positive definite.

Matrix :

is positive definite.

Matrix :

✓
✓
(expanding along row 2) ✓

is positive definite.

Answer: All three matrices , , are symmetric positive definite.